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Recommended Books and Resources 

This lecture course covers three topics: kinetic theory, stochastic processes and hnear 
response. Most decent books on statistical mechanics will have a section covering non- 
equilibrium topics in general. However, if you're looking for details beyond the basics, 
you'll probably need a different book for each topic. Some good general purpose books 
are: 

• Huang, Statistical Mechanics 

• Kardar, Statistical Physics of Particles 

• Reif , Fundamentals of Statistical and Thermal Physics 

Both Huang and Kardar treat kinetic theory and the Boltzmann equation before they 
move onto statistical mechanics. Much of Section 2 of these notes follows the path laid 
down in the these books. Reif ends with a much wider ranging discussion of kinetic 
theory, transport and stochastic processes. 

For more details on kinetic theory: 

• Chapman and Cowling, The Mathematical Theory of Non- Uniform Gases 

• Lifshitz and Pitaevskii, Physical Kinetics 

Both of these are old school. The first was published in 1939 although the latest edition, 
written in 1970, is modern enough to cover all the developments that we touch upon 
in this course. The last volume of the course by Landau and Lifshitz covers kinetic 
theory. This book was written substantially later than the earlier volumes, decades 
after Landau's death. 

For more details on stochastic processes: 

• Van Kampen, Stochastic Processes in Physics and Chemistry 

The topic of linear response is usually covered in books on many body theory or more 
general condensed matter. Two excellent modern books, both with a chapter on re- 
sponse theory, are 

• Altland and Simons, Condensed Matter Field Theory 

• Chaikin and Lubensky, Principles of Condensed Matter Physics 

Finally, there are a number of good lecture notes and resources on the web, collated at 
http: / / www.damtp.cam.ac.uk/user/tong/kinetic.html 
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1. Things Bumping Into Other Things 



1.1 Introduction 

The purpose of this course is to describe a number of basic topics in non-equihbrium 
statistical mechanics. 

If you've taken a first course in statistical mechanics, you'll know that the whole 
machinery of ensembles and partition functions only works when applied to systems 
in equilibrium. Equilibrium is defined to be a state in which, at least on the coarse 
grained level, things don't change. Of course, if you have a hot system and you look 
closely enough, everything is flying around on the atomic level. But if you focus only 
on macroscopic variables then, in equilibrium, all is calm. 

At first, the restriction to equilibrium sounds rather limiting. But it's not. This is 
because the state of equilibrium is very special: if you take any system and wait long 
enough then it will eventually relax down to equilibrium. (This is sometimes said to 
be the —1**^ law of thermodynamics). 

Of course, this begs the question of why equilibrium is special. Why do all systems 
eventually reach this state. How do they approach this state? How does such irre- 
versible behaviour arise from the fundamental laws of physics which are, for all intents 
and purposes, invariant under timer reversal? Moreover, what if you're not happy to 
just sit back and watch an equilibrium system? Suppose you want to stir it or splash 
it or attach a couple of crocodile chps and zap it. How will it respond? These are the 
kind of questions that we will begin to answer in this course. 

While there is typically only a single equilibrium state, for a system with 10^^ par- 
ticles, there are many many ways to be out-of-equilibrium. Most of these states are 
uninteresting in the sense that they will be so complicated that no general features will 
emerge. Moreover, such states will be fleeting, rapidly changing to another complicated 
configuration. If we're to have any chance of making progress, we need to be careful 
about the kind of states we discuss and the kind of questions that we ask. We would 
like to identify features in the dynamics of 10^^ particles that persist for long periods 
of time. We will see that such features arise for systems that are close to equilibrium. 
Indeed, throughout this course, the dramatic sounding "non-equilibrium" will really 
mean "almost-equilibrium" . 

Each of the four sections in these lecture notes can be read more or less independently. 
In the rest of this introductory section, we will introduce a few basic tools to describe 
how quantities change in a gas. This will really be a baby version of kinetic theory, with 
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nothing more sophisticated than Newtonian thinking apphed to a bunch of bilhard balls. 
But it will allow us to develop some basic intuition for the rudiments of the subject. 
While many of the formulae we derive in this section are rather heuristic, all will be 
revisited Section 2 where we use the Boltzmann equation to give a more rigorous view 
on the subject, understanding transport phenomena and deriving the equations of fluid 
mechanics starting from first principles. Section 3 introduces the subject of random 
jittery motion, usually called stochastic processes. Finally, in Section 4 we turn the 
stir-it-splash-it-zap-it question and develop the machinery necessary to describe how 
systems respond when prodded. 

1.2 Basics of Collisions 

Let's start by considering molecules in a gas of volume V. We will begin by ignoring 
all interactions between particles. Instead, we will treat the molecules as spheres of a 
finite size which will allow collisions to occur. For the most part, we won't rely on the 
results of earlier courses on statistical mechanics. There is, however, one exception: in 
the rest of this section, we will need the Maxwell-Boltzmann probability distribution 
for the velocities in a gas^. 

^^^^^ '^'^ ^ (2;^^) ^"'""'^''"^'^'^ (1-1) 

The distribution f{v)cPv is the probability that a molecule has velocity within a small 
volume d^v in the neighbourhood of v. 

We denote the diameter of the particle as d. Obviously its radius is d/2. Viewed 
head on, the particle appears as a disc with area 7r((i/2)^. However, more relevant for 
our purposes is the effective cross-sectional area of the particle, nd^. To see why this is, 
focus on a single particle as it makes its way through the gas. If it travels a distance 
it will sweep out a volume nd^l as shown in Figure 1 and collide with any other particle 
whose centre lies within this volume. 

The m,ean free path is defined to be the average distance travelled by the molecule 
between each collision. This is given by Tid^l = V/N, or 



Trd'^ mrd'^ 

where n = N/V is the particle density. 

^This result will be rc-dcrivcd in Section 2 when wo discuss the Boltzmann equation. You can 
also find a simple derivation in any book or lecture notes on statistical physics. See, for example, 
http: / / www.damtp.cam.ac.uk/ user/tong/ statphys.html 
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Figure 1: A particle of radius d/2 travels, on average, a length I between each collision. In 
this time it sweeps out a volume ircPl. 

In what follows, we'll assume that our gas is dilute, meaning / ^ d. For typical gases 
d ~ 10~^^m while, at atmospheric pressure, / ~ 10~^m. 

1.2.1 Relaxation Time 

The average time between collisions is called the scattering time or relaxation time, 

I 

Vrel 

You might think that Vrei is the average speed of a given particle. This isn't quite true. 
Since we're interested in the rate of collisions, the speed of other particles approaching 
is just as important as the speed of the particle you're looking at. So we should take 
Vrel to be the average relative speed of the molecules. For two particles with velocities 
V and v', the average relative speed is 

vL = {{v-v'Y) = Jd'vj d'v' {v-v'ff{v)f{v') 

= {v^) + {v'^)-2{v-v') (1.3) 

where f{v) in the first line is the Maxwell-Boltzmann distribution (1.1). The fact that 
we have multiplied the distributions f{v)f{v') together in the first line means that 
we are assuming that the velocities of the two particles are uncorrelated. This is an 
assumption that we shall return to in Section 2. 

The last term in (1.3) vanishes: {v ■ v') = 0. This follows on rotational grounds. 
Because the velocity of each particle is independent, it's enough to know that the 
average velocity (not speed!) in, say, the x-direction vanishes: (vx) = 0. Meanwhile, 
(v^) = (f'^) which means that v^^i = 2(f ^). It is a simple exercise to compute (w^) from 
the Maxwell-Boltzmann distribution (1.1) and the answer is the same as you would get 
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by simply appealing to the equipartition of energy: (v^) — 3kBT/m. We have 

_2 ^ksT 
m 

and the relaxation time is given by 

1 I m 

Notice that as the temperature drops, the mean free path remains unchanged. However, 
the time between collisions increases. 



There is a slightly different interpretation of the relaxation time that it is useful 
to have in hand. Suppose that the probability that a molecule undergoes a collision 
between time t and time t -\- dt is given by wdt, for some constant known as the 
collision rate. Notice that in stating this, we have made more assumptions about the 
nature of the collisions. In particular, the fact that w is a constant means that no 
memory of previous collisions is kept: the chances of being hit again are not affected 
just because you already were hit a short time ago. 

If P{t) is the probability that the molecule makes it to time t unharmed, then the 
probability that it further makes it to time t + dt without collision is 



P{t + dt) ^P{t){l-wdt) 

Writing this as a differential equation, we have 

dP 

= -wP =^ P(t) = we-"'* 

dt 

where we've chosen to fix the normalization so that P(t)dt = 1 which guarantees 
that no particle lasts forever without undergoing a collision. With this in hand, we can 
compute the average time between collisions. But this is exactly the quantity that we 
called the relaxation time above. It is 



poo 

T = / tP{t)dt^ 
Jo 

We learn that l/r is the coUision rate. 



1 

w 
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1.3 Basics of Transport 

We now turn to the question of how things move. Of course, in a thermal system, 
the microscopic constituents are alway moving, even in equihbrium. Our goal here 
is to understand how certain macroscopic properties move when out of equilibrium. 

The properties that we will look at are all associated to a conserved quantity: particle 
number, energy and momentum. Processes in which these quantities change over time 
are usually referred to as transport. As we will see, all of these quantities typically flow 
in such a way as to reach the equilibrium state. 

1.3.1 Diffusion 

Drop a blob of ink into a glass of water. How does it spread? More generally, we are 
interested in the motion of a particular kind of particle - one with a nice colour, or 
funny smell - as it makes its way through a generic background of liquid or gas. The 
true dynamics of any particle is, as you might expect, somewhat jittery. Here we'll look 
at a simple model that captures this physics. 

Random Walk 

Consider a lattice which, for now, we take to be one dimensional. The spacing between 
the lattice sites is set by the mean free path, /, and after a time, r, the particle jumps 
either left or right. The direction of the jump is entirely random: 50% of the time it 
goes left, 50% right. This model is known as a random walk. 

The particle starts at the origin and we want to know the probability P{x, t) that it 
sits at X = ml at time t = Nr. (Here m is an integer; it's not the mass of the particle!). 
We'll start by giving a simple combinatoric derivation of the answer. For simplicity, 
we'll take N to be even and we'll look at m ^ A^. To get to x = ml, the particle must 
have made |(A^ + m) forward jumps and ^{N — m) backwards jumps. The probability 
is just the number of different ways we can do this, divided by 2^, the total number of 
possible combinations. 

where, in the second step, the factorials have been replaced by Stirling's approximation 
and we've also expanded to leading order in m/N . (To get the prefactor, we need to 
go to the third order in the Stirling expansion). 
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The probability distribution of the particle is an ever-spreading Gaussian ensemble. 
The mean is simply (x) = 0, reflecting the fact that the particle is equally likely to 
travel forwards as backwards. The variance is 

{x')^^^t (1.5) 

The root-mean-square (rms) distance travelled by the particle grows as -s/Jx^ ~ \/t. 
This is characteristic behaviour of random walks. 

It is simple to repeat our analysis of the random walk to three dimensions. For a 
cubic lattice, we assume that the motion in each of the directions is independent and 
equally likely. On average, the particle moves in the x-dircction only every 3t, so (1.5) 
should be replaced by {xP') = IH/3t. But this means that the total rms distance covered 
remains unchanged 

{x') = {x') + {y') + {z') = 

The Diffusion Equation 

We can recast the above discussion in terms of a differential equation for the density 
of particles, n = N/V. Away from equilibrium, the density is not a constant. It is, in 
general, a function of time and space. We expect any gradient, Vn, in the density of 
particles to lead to a flow, from the high density region to the low. 

We'll again restrict flrst to the case of one-dimension. Consider the density at some 
fixed time: n = n{x,t). We'd like to derive an expression for the density at the point 
X a short time At later. Of course, some particles will leave, but others will come in 
to replace them. Any particle which is at x at time t -\- At must have been sitting at 
some other position x — Ax at time t. Here Ax should be viewed as a random variable 
since some move one way, some the other. This means that we can write an expression 
for the density at time i -|- At as an average over all the different Ax, 

n{t + At, x) = {n{t, X - Ax)) 

dfi 1 d^Ti 

= n(t.x)-^(A.) + -^(Ax^) + ... 

The term with the first order derivative vanishes because, on average, particles are 
equally likely to go either way, meaning (Ax) = 0. Taylor expanding the left-hand- 
side, we arrive at the diffusion equation 

dn ^ d^n 
dt dx^ 
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where the diffusion constant is D = (Aa;^)/2At. We expect this to be related to our 
two quantities, the mean free path / and scattering time r. On dimensional grounds, 
we must have 



Solutions to the diffusion equation evolve so as to iron out any inhomogeneities in 
particle density. As an example, suppose that all N particles start out life sitting at 
the origin, giving us the initial condition n{x,t = 0) = Nd{x). The solution to the 
diffusion equation with this initial condition is an ever-spreading Gaussian, 



This reproduces the discretised result (1.4). Viewing the average distance travelled as 
the width of the cloud of particles, we again have the result 



It is simple to extend the derivation above to three dimensions. Going through the 
same steps, we now find the 3d diffusion equation. 



This is also known as Pick's (second) law. We again expect that D ~ /^/r. (Although 
the overall numerical factor is not necessarily the same as the Id case. In fact, in simple 
analysis it is a factor of 3 less). The Gaussian again provides a solution, now with 



As we will now show, a number of other processes also follow this general diffusive 
form. 

1.3.2 Viscosity 

Viscosity is a form of internal friction experienced by a fluid. It can be measured 
by placing a fluid between two plates, a distance d apart in the z direction. Holding the 
lower plate stationary, the top plate is moved at a constant speed, u, in the x direction. 
But you don't get to do this for free: the fluid pushes back. If you want to keep the 
plate moving at a constant speed, you have to apply a force F. 





{x^) = 2Dt 



{x^) = 6Dt 
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Near the upper plate, a friction force causes the 
fluid to be dragged along with the same speed u. How- 
ever, near the lower plate, the fluid remains stationary. 
This sets up a velocity gradient, Ux{z), with Ux{d) = u 
and Ux{0) = 0. Experimentally, it is found that the force 
per unit area which must be exerted on the upper plate 
is proportional to this velocity gradient, 

F dUx u 
A ^ dz ^ d 

where the second equality holds for small distances d. The coefficient of proportionality, 
r], is called the viscosity. (Or, more correctly, the dynamic viscosity). 

We would like to derive both the force law (1.6) and the viscosity t] from first princi- 
ples. It's simple to get an intuition for what's happening on the atomic level: when the 
molecules collide with the upper plate, they pick up some x-momentum. They then 
collide with other molecules lower down, imparting some of this x-momentum to new 
molecules, which then collide with other molecules lower down, and so on. In this way, 
we set up the velocity gradient in the z direction. 

We'll think of a slab of gas at some fixed value of z. To figure out the force acting on 
this slab, we need to work out two things: the number of particles moving through the 
slab per unit of time; and the extra momentum in the x-direction that each particle 
imparts to the molecules in the slab. 

Let's first deal with the number of particles. The density of particles in the fiuid is 
n = N/V. How many of these pass through a slab in the 2;-direction in a given length 
of time depends on how fast they're travelling in the z-direction (obv!). But we know 
how many particles there are with each speed: this is given by the Maxwell-Boltzmann 
distribution (1.1). The net result is that the number of particles, per unit time, per 
unit area, whose velocity is lies close to v (in a box of size d^v), passing through a 
horizontal slab is 



# of particles per unit time per unit area = nvz f{v) d^v (1.7) 

Now let's figure out the momentum that each of these molecules imparts. Consider a 
particle at some position z. It gets hit from below, it gets hit from above. The hits 
from above are likely to give it more momentum in the x direction; those from below, 
less. Let's consider those ariving from above. If they arrive from a position z -|- Az, 
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then they impart x-momentum 



Ap — m{ux{z + Az) — Ux{z)) 



du. 



'X 



Az 



(1.8) 



~ m 



dz 



What is the distance Az here? Well, this depends 
on the angle the particles come in at. They have 
travelled the mean free path I, so if they arrive at 
angle 9 then we must have 




/ COSQ 



Figure 3: 



Az = I cos 9 



Here 9 e [0, 7r/2) for particles arriving from above. But the same argument also holds 
for particles coming in from below. These have 9 e (7r/2,7r] and, correspondingly, 
Az < which, from (1.8), tells us that these particles typically absorb momentum 
from the layer at z. 

Our goal is to work out the force per unit area acting on any z slice. This is given 
by the rate of change of momentum 



where the minus sign arises because F defined in (1.6) is the force you need to apply to 
keep the fiow moving (while Ap/At is the force of the fiuid pushing back). The rate of 
change of momentum per unit area is simply the product of our two expressions (1.7) 
and (1.8). We have 



We've actually done something sly in this second line which is not really justified. 
We're assuming that the fluid has an average velocity (vx) = Ux in the a;-direction. 
Yet, at the same time we've used the Maxwell-Boltzmann distribution for the velocity 
of the particles which has (vx) = 0. Presumably this is not too bad if the speed of the 
flow M -C {v), the average speed of the particles in the fluid, but we really should be 
more careful in quantifying this. Nonetheless, the spirit of this section is just to get a 



F 
~A 



1 Ap 



F 
~A 




9 



heuristic feel for the physics, so let's push on regardless. Writing the velocity integral 
in polar coordinates, we have 



= —mn 



A dz 



J dvv^ J d9 sine J dcj) {-v cos 9)1 cos 9 {—^j; j e^^sT 9) 

At this stage we can trivially do the J d(f) integral and d9 cos'^ 9 sin 9 — 2/3. We're 
left with 



F mnl dux f , . c / m 



3/2 



dv^^vH-^] e-^-'l- (1.10) 



A 3 dz J \2nkBT ^ 

But the integral / dv is simply the expression for the average speed {v) in the gas. We 
have our final expression, 

F 1 , , , dur 



— — -mnl{v) 



A3 ' ' dz 
Comparing with (1.6), our expression for the viscosity is 

77 = -mnl{v) (1-11) 

There is something surprising about the viscosity: it is independent of the density 
n — N/V of the gas. At first sight that looks like a wrong statement because, obvi- 
ously, there is a factor of n sitting in (1.11). But remember that the mean free path 
depends inversely on the density, / ~ 1/n, as we can see from (1-2). The fact that the 
viscosity docs not depend on the fluid density is rather counterintuitive. You might 
think that denser gasses should be more viscous. But the derivation above provides 
the explanation for this behaviour: if you halve the density, there are half as many 
molecules moving down. But each travels twice as far and therefore imparts twice the 
momentum kick Ap when they finally hit. 

The expression (1.11) holds a special place in the history of physics. It was first 
derived by Maxwell and is arguably the first truly novel prediction that was made 
using kinetic theory, providing important evidence for the existence of atoms which, at 
the time, were not universally believed. Indeed, Maxwell himself was surprised by the 
fact that 77 is independent of the density of the gas, writing at the time 

''Such a consequence of the mathematical theory is very startling and the 
only experiment I have met with on the subject does not seem to confirm 
if. 

Maxwell rose to the challenge, building the apparatus and performing the experiment 
that confirmed his own prediction. 
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1.3.3 Thermal Conductivity 

The next transport process we will look at is the conduction of heat. Place a fluid 
between two plates, each held at a different temperature. Empirically, one finds a flow 
of energy in the fluid. This is described by the heat flow vector, g, defined by the energy 
per unit time passing through a unit area (which is perpendicular to q). Empirically, 
the flow of heat is proportional to the temperature gradient. 



where k is called the thermal conductivity. Once again, we would like to derive both 
this empirical law, as well as an expression for k. 

Our calculation follows the same path that we took to determine the viscosity. Let's 
set up a temperature gradient in the z-direction. The number of particles with velocity 
V that pass through a slab at position z per unit time per unit area is again given 
by (1.7). We'll use equipartition and assume that the average energy of a particle at 
position z is given by 



We also need to know how particles deposit or gain energy when they reach the slab. If 
a particle came from a hot place with temperature T(z + Az), we'll assume the particle 
deposits the difference in energy. Similarly, if the particle arrives from a colder place, 
we'll assume it absorbs the difference. This means 



Recall that the height Az from which the particle arrives depends on both the mean 
free path and the angle at which it comes in: Az = / cos 9. 

As in the derivation of the viscosity, there is something a little dodgy in what we've 
written above. We've equated the energy deposited or gained by a particle with the 
average energy. But this energy transfer will certainly depend on the velocity of the 
particle and which is dictated by the Maxwell-Boltzmann distribution in (1.7). As in 
the derivation of the viscosity, we will simply ignore this fact and proceed. We'll do 
better in the next section. 

Modulo the concerns above, we now have enough information to compute the heat 
flow. It is 



-kVT 



(1.12) 



E{z) = \kBT{z) 
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Doing the integrals J d^v using the same steps that took us from (1.9) to (1.10), we 
derive the law of heat flow (1.12) 

The thermal conductivity is the proportionality constant. It is usually expressed in 
terms of the specific heat, cy, of the ideal gas 

« = lcvl{v)) (1.13) 



where 



Cv = ^nkB 



1.3.4 Conservation Means Diffusion 

Thermal conductivity is all about the transport of energy; viscosity is about the trans- 
port of momentum. But both energy and momentum have a very special property: 
they are conserved. 

What's more, because physics is local, we can make a stronger statement than just 
"the total energy doesn't change". If the energy in some region of space, E{x), changes 
then it must show up in a neighbouring region of space. But that's exactly what the 
heat flow q is telling us: how energy is moving from one point to the next. This local 
conservation law is captured by the equation. 

Once again equating energy with the thermal energy, E{x) — |A;bT'(x), the continuity 
equation reads 

^ = -lv-g=--V2T (1.14) 
at Cv Cv 

This is the heat equation. It tells us that any inhomogeneities in temperature are 
smoothed out through diffusion with diffusion constant D — k/cv — \l{v) ~ /^/r. 

There is a similar story for momentum, where i = 1, 2, 3 labels the three directions 
of space. The continuity equation reads 



d,t dxJ 







where P-'* is the pressure tensor which describes the flux of i-momentum in the j- 
direction. 
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But looking back at our derivation of the viscosity in Section 1.3.2, this is precisely 
what we equated to the force F/A\ the flux of x-momentum in the ^-direction. (Ac- 
tually there's an extra minus sign that follows from our previous definition of F). 
Combining the continuity equation with our earlier expression for the viscosity, we find 

dp^ du,j. d'^Ux 



where, as in Section 1.3.2, we've restricted to situations with no velocity gradients in the 
X and y directions. The result is once again a diffusion equation, this time for gradients 
in velocity. And, once again, the diffusion constant given hy D = rj/mn = ^l{v) ~ P/t. 

We learn that all roads lead to diffusion. For any conserved quantity - whether par- 
ticle number, energy or momentum - any inhomogeneities in the system are smoothed 
away through the diffusion equation. 

The equations that we've written down in this final section are rather hand-waving 
and, in cases, missing some interesting physics. The proper equations are those of 
hydrodynamics. The goal of the next section is the do a better job in deriving these. 
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